Update docs and yaml for baselines #292
Conversation
WalkthroughThe recent updates enhance the replication guide by providing detailed instructions for reproducing study baselines, adapting fine-tuned models for various tasks, and conducting inference with updated methodologies. These changes aim to streamline the process of model replication, adaptation, and evaluation, making it more accessible and efficient for users by offering clear, step-by-step guidance and emphasizing the use of specific configuration notes and YAML files. Changes
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Review Status
Actionable comments generated: 9
Configuration used: CodeRabbit UI
Files ignored due to path filters (14)
experiments/baselines/med3d/extract_features.yamlis excluded by:!**/*.yamlexperiments/baselines/med3d/finetune.yamlis excluded by:!**/*.yamlexperiments/baselines/med3d/supervised_training/get_predictions_task1.yamlis excluded by:!**/*.yamlexperiments/baselines/med3d/supervised_training/get_predictions_task2.yamlis excluded by:!**/*.yamlexperiments/baselines/med3d/supervised_training/get_predictions_task3.yamlis excluded by:!**/*.yamlexperiments/baselines/models_genesis/extract_features.yamlis excluded by:!**/*.yamlexperiments/baselines/models_genesis/finetune.yamlis excluded by:!**/*.yamlexperiments/baselines/models_genesis/supervised_training/get_predictions_task1.yamlis excluded by:!**/*.yamlexperiments/baselines/models_genesis/supervised_training/get_predictions_task2.yamlis excluded by:!**/*.yamlexperiments/baselines/models_genesis/supervised_training/get_predictions_task3.yamlis excluded by:!**/*.yamlexperiments/baselines/supervised_training/supervised_finetune.yamlis excluded by:!**/*.yamlexperiments/baselines/supervised_training/supervised_random_init.yamlis excluded by:!**/*.yamlexperiments/inference/extract_features.yamlis excluded by:!**/*.yamlexperiments/inference/get_predictions.yamlis excluded by:!**/*.yaml
Files selected for processing (3)
- docs/replication-guide/baselines.md (1 hunks)
- docs/replication-guide/fm_adaptation.md (1 hunks)
- docs/replication-guide/inference.md (2 hunks)
Additional comments: 6
docs/replication-guide/fm_adaptation.md (3)
- 9-9: The instruction emphasizes searching for '
Note:comments. To improve clarity and consistency, consider specifying the exact nature of these comments more clearly, e.g., by providing an example or explaining what kind of changes these comments will guide the user to make.- 7-7: There's a possible spelling mistake detected near line 7. However, upon review, it seems to be a false positive related to the term "init" in "random init baseline," which is a valid term in this context.
- 6-12: > 📝 NOTE
This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [27-27]
There's an unpaired symbol near line 27. Ensure that all markdown links are correctly formatted with both opening and closing brackets.
docs/replication-guide/inference.md (1)
- 13-13: There's an unpaired symbol near line 13. Ensure that all markdown links are correctly formatted with both opening and closing brackets.
docs/replication-guide/baselines.md (2)
- 7-7: The term "init" in "random init baseline" is correctly used in this context, addressing the static analysis tool's flag as a false positive.
- 26-26: The term "Med3D / MedicalNet" is correctly spelled, addressing the static analysis tool's flag as a false positive.
|
|
||
| We provide the YAML configuration for this at `experiments/adaptation/fmcib_finetune.yaml`. | ||
|
|
||
| By default, we configure this for Task 1. You can adapt this for Task 2 and Task 3 by searching for 'Note: ' comments in the YAML that outline what must be changed. Make sure you download the weights for the pre-trained foundation model before attempting to reproduce this training. | ||
| By default, we configure this for Task 1. You can adapt this for Task 2 and Task 3 by searching for `Note: ` comments in the YAML that outline what must be changed. Make sure you download the weights for the pre-trained foundation model before attempting to reproduce this training. | ||
|
|
||
|
|
||
| You can start training by running this in the root code folder, |
There was a problem hiding this comment.
📝 NOTE
This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [18-18]
The term "hyper-parameter" is typically spelled as one word: "hyperparameter." Consider updating this for consistency with common usage in machine learning documentation.
- hyper-parameter optimization framework
+ hyperparameter optimization framework📝 NOTE
This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [19-19]
The phrase "In order to" can be simplified to "To" for conciseness without losing meaning.
- In order to perform the modelling, you can run
+ To perform the modelling, you can run📝 NOTE
This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [31-31]
Consider reducing the number of consecutive spaces before "using --scoring roc_auc" for consistency and readability.
- for instance, using `--scoring roc_auc`
+ for instance, using `--scoring roc_auc`📝 NOTE
This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [33-33]
The phrase "to try our the modelling process" seems to contain a typo. It should likely be "to try out the modelling process."
- to try our the modelling process
+ to try out the modelling process|
|
||
| ## Running predictions from our supervised models (Finetuned FM/ Baselines) | ||
| However, this method might be preferred when features need to be extracted from different models (used as baselines in our study). Follow the `Note:` in the corresponding config file to change model paths and use different baselines tested. |
There was a problem hiding this comment.
To enhance clarity and readability, consider rephrasing "However, this method might be preferred when features need to be extracted from different models" to start with an uppercase letter and simplify the sentence structure.
- However, this method might be preferred when features need to be extracted from different models
+ However, this method is useful for extracting features from various modelsCommittable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.
| However, this method might be preferred when features need to be extracted from different models (used as baselines in our study). Follow the `Note:` in the corresponding config file to change model paths and use different baselines tested. | |
| However, this method is useful for extracting features from various models (used as baselines in our study). Follow the `Note:` in the corresponding config file to change model paths and use different baselines tested. |
|
|
||
| ## Running predictions from our supervised models (Finetuned FM/ Baselines) | ||
|
|
||
| To run predictions from our models (both supervised and self-supervised), we provide YAML files that can be run with the lighter interface. These are found in `experiments/inference`, namely `get_predictions.yaml` for getting the predictions. | ||
|
|
There was a problem hiding this comment.
There's a spelling mistake in "Beofre" which should be corrected to "Before."
- Beofre running the predictions config, if you haven't downloaded the `models` folder contraining all our baselines, you can do so using
+ Before running the predictions config, if you haven't downloaded the `models` folder containing all our baselines, you can do so usingCommittable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.
| !!! note | ||
| While the above pipeline will allow you to extract features, we provide an easier and simpler, recommended API to do this. Please refer to [Quick Start](../getting-started/quick-start.md) or [Cloud Quick Start](../getting-started/cloud-quick-start.md) | ||
|
|
||
| However, this method might be preferred when features need to be extracted from different models (used as baselines in our study). Follow the `Note:` in the corresponding config file to change model paths. | ||
|
|
||
| ## Running predictions from our supervised models (Finetuned FM/ Baselines) | ||
| However, this method might be preferred when features need to be extracted from different models (used as baselines in our study). Follow the `Note:` in the corresponding config file to change model paths and use different baselines tested. | ||
|
|
||
| ## Running predictions from our supervised models (Finetuned FM/ Baselines) | ||
|
|
||
| To run predictions from our models (both supervised and self-supervised), we provide YAML files that can be run with the lighter interface. These are found in `experiments/inference`, namely `get_predictions.yaml` for getting the predictions. | ||
|
|
There was a problem hiding this comment.
📝 NOTE
This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [22-22]
The word "contraining" seems to be a typo. Correct it to "containing."
- contraining all our baselines
+ containing all our baselines| @@ -37,3 +37,5 @@ lighter predict --config_file ./experiments/inference/get_predictions.yaml | |||
| ``` | |||
| As with the previous YAMLS, please follow the 'Note:' tags to place appropriate data paths and change relevant parameters. This YAML is to be used if you want to get target predictions from the models. | |||
There was a problem hiding this comment.
The term "YAMLS" is not standard and could be confusing. Consider using "YAML files" for clarity.
- As with the previous YAMLS, please follow the 'Note:' tags
+ As with the previous YAML files, please follow the 'Note:' tagsCommittable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.
| As with the previous YAMLS, please follow the 'Note:' tags to place appropriate data paths and change relevant parameters. This YAML is to be used if you want to get target predictions from the models. | |
| As with the previous YAML files, please follow the 'Note:' tags to place appropriate data paths and change relevant parameters. This YAML is to be used if you want to get target predictions from the models. |
| @@ -1,2 +1,46 @@ | |||
| # Reproduce Baselines | |||
| :hourglass_flowing_sand: Coming soon! :hourglass_flowing_sand: | |||
|
|
|||
| Reproducing baselines used in this study is very similar to the adaptation of the FM as essentially we are just using the FM weights for adaptation. | |||
There was a problem hiding this comment.
Consider adding a comma after "FM" and "essentially" for better readability and to improve the flow of the sentence.
- Reproducing baselines used in this study is very similar to the adaptation of the FM as essentially we are just using the FM weights for adaptation.
+ Reproducing baselines used in this study is very similar to the adaptation of the FM, as essentially, we are just using the FM weights for adaptation.Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.
| Reproducing baselines used in this study is very similar to the adaptation of the FM as essentially we are just using the FM weights for adaptation. | |
| Reproducing baselines used in this study is very similar to the adaptation of the FM, as essentially, we are just using the FM weights for adaptation. |
Description
Related Issue
Type of Change
Checklist
CODE_OF_CONDUCT.mddocument.CONTRIBUTING.mdguide.make codestyle.Summary by CodeRabbit